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He  deal  with  the  rounding  error  analysis  of  successive  approximation 

iterations  for  the  solution  of  large  linear  systems  Ax  ■ b.  We  prove  that 

Jacobi,  Richardson,  Gauss-Seidel  and  SOR  iterations  are  numerically  stable 
* 

whenever  A ■ A >0  and  A has  Property  A.  This  means  that  the  computed 
result  x^  approximates  the  exact  solution  or  with  relative  error  of  order 
ClKII  * IK”1  II  where  C is  the  relative  computer  precision.  However  with  the 
exception  of  Gauss-Seidel  iteration  the  residual  vector  IKvbH  18  of  order 
clMflh'1!!  IMI  and  hence  the  remaining  three  iterations  are  not  well-behaved. 


II 


1.  INTRODUCTION 


This  paper  deals  with  the  rounding  error  analysis  in  floating  point 
arithmetic  of  successive  approximation  iterations  for  the  solution  of  large 
sparse  linear  systems  Ax  ■ b. 

We  summarize  the  results  of  this  paper.  Basic  concepts  of  numerical 

stability  and  good-behavior  are  recalled  in  Section  2.  We  give  necessary 

and  sufficient  conditions  for  numerical  stability  and  good-behavior  in 

Sections  3 and  5.  In  Section  4 we  deal  with  several  examples  of  successive 

approximation  Iterations.  We  prove  that  Jacobi,  Richardson,  Gauss-Seldel 

* 

and  SOR  iterations  are  numerically  stable  whenever  A ■ A >0  and  A has 
Property  A.  In  Section  6 we  show  that  with  the  exception  of  Gauss-Seldel 
iteration  they  are  not  well-behaved.  In  the  last  section  we  indicate  that 
good-behavior  of  any  numerically  stable  method  can  be  achieved  by  the  use 
of  iterative  refinement  even  if  all  computations  are  performed  in  single 
prec  is  ion . 
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2.  PRELIMINARIES 

In  this  section  we  briefly  recall  what  we  mean  by  numerical  stability 
and  good-behavior  of  an  iteration  for  solving  a linear  system  Ax  ■ b where 
A is  a n X n nonsingular  complex  matrix  and  b is  a n X 1 vector.  Ue  shall 
assume  throughout  this  paper  that  ||*||  denotes  the  spectral  norm. 

Let  {x^}  be  a computed  sequence  of  successive  approximations  of  the  solu- 
tion a ■ A*^  by  an  iteration  tp  in  t digit  floating  point  arithmetic  fl,  see 
Wilkinson  [63]. 

An  iteration  cp  is  called  numerically  stable  if 

(2.1)  Tto  |L  - o||  * £c  cond(A)  ||  cr||  + 0(C2) 

k K 1 

where  £ * 2-t  is  the  relative  computer  precision,  is  a constant  which 
depends  only  on  the  size  n of  the  problem,  and  cond(A)  * J|  A f|  • ||  A *||  is 
the  condition  number  of  A. 

An  Iteration  cp  is  called  well-behaved  (or  equivalently  9 has  good- behavior) 
if 

(2.2)  ITS  ||  Axk  - b||  s £c2  ||  A||  ||  a||  + 0(£2) 
where  * c2 (n) . 

It  is  easy  to  verify  that  good-behavior  implies  numerical  stability  but 
not,  in  general,  vice  versa.  Furthermore,  cp  is  well-behaved  iff  there  exist 
matrices  such  that  for  large  k 

(2.3)  (A  + Ek)  xk  - b and  ||  eJ|  * £c3  ||  A ||  + 0(£2) 
for  c3  - c3(n) . 


T 
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Thus  good-behavior  means  Chat  x^  Is  Che  exact  solution  of  a slightly 
perturbed  system  or  equivalently  that  the  residual  vector  r^  " Axk  " b 18 
small  in  the  sense  of  (2.2). 

Recall  that  commonly  used  direct  methods  such  as  Gaussian  elimination 
with  pivoting.  Householder  method,  modified  Gram-Schmidt,  or  Gram-Schaidt 
with  reorthogonalization  are  well-behaved.  Let  us  also  mention  that  Chebyshev 
iteration  is  numerically  stable  but,  in  general,  is  not  well-behaved;  see 
Wozniakowski  [75]  where  a detailed  discussion  of  these  concepts  may  be  found. 


! 
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3.  NUMERICAL  STABILITY  OF  SUCCESSIVE  APPROXIMATION  ITERATIONS 
We  consider  the  numerical  solution  of  a large  linear  system 

(3.1)  Ax  - b 

where  A is  a nonsingular  complex  n x n matrix  and  b is  a n x 1 complex  vector. 
We  assume  that  A is  a sparse  matrix  of  high  order  and  nr  ■ A is  the  solution 
of  (3.1). 

A successive  approximation  iteration  is  defined  as  follows: 

(i)  Transform  Ax  ■ b to  an  equivalent  system 

(3.2)  x ■ Hx  + h,  (Q  - Ho  + h) . 

Sometimes  H ■ H(A)  is  chosen  to  minimize  the  spectral  radius  a(H)  of  H, 
a(H)  <1,  in  a certain  class  of  {H(A) }. 

(ii)  Solve  (3.2)  by  the  iteration 

(3.3)  xfc+1  - Hxfc  + h,  k - 0,1,... 

where  Xq  is  a given  initial  approximation. 

Using  different  transformations  we  get  different  iterations;  see  Section 
4 where  Jacobi,  Richardson,  Gauss-Seidel  and  successive  overrelaxation  (SOR) 
iterations  are  considered. 

Let  e^  ■ x^  - a.  From  (3.3)  we  get  the  theoretical  error  formula 

(3.4)  ek-Hke0. 


p ( 

11 


Thus  the  theoretical  Iteration  is  convergent  for  any  Xq  iff  the  spectral 
radius  o(H)  is  less  than  1.  Furthermore  the  character  of  convergence  mainly 
depends  on  9(H)  since 


lim 


IK  || 


k (9(H)  + «) 


for  any  e > 0. 


Due  to  the  sparseness  of  A in  many  cases  we  can  compute  the  product  Hx^ 

2 


and  x^+j  in  time  and  storage  proportional  to  n rather  than  n . However  in 
floating  point  arithmetic  fl  we  cannot  compute  Hx^  or  x^+1  from  (3.3)  exactly. 
Assume  that 


> 


l 


r j 
l 

( I 


i 


(3.5)  fldb^  + h)  - (H  + ay  x^  + (I  + 6lk)  h - Hxk  + h + §k 
where  ||  SHjJI  s gCjJI  H |j,  ||  fi Ik |f  ^ Cj  and  c 2 depend  only  on  n and 

(3.6)  5k  - Xj^  + 6^(1  - H)  ». 

Note  that  (3.5)  holds  for  most  algorithms  used  in  numerical  practice  with  c^ 
and  c 2 of  order  unity. 

Thus,  instead  of  the  theoretical  relation  (3.3)  we  get 
(3.?)  xk+1  - Hxk  + h + ?k.  > 

It  follows  that  the  error  formula  for  the  computed  sequence  ek  ■ xk  - a is 
equal  to 

k 

/■>  o\  - .»k+l  , „k-i  _ 

(3*8)  *k+l  H e0  + H V 

i-0 

compare  with  (3. A). 

I 

1 


f 


6- 


From  (3.5)  and  (3.6)  the  vectors  g^  have  a bound 

(3-9)  ||Sk||  * Cc3(||h||  + ||  I - H||  )||  «||+  CCjII^  - all 

for  c3  - max (c  ^ , c2) . 

Let  {71  ) be  a sequence  such  that  ||  ||  s 1.  Define 

k 

(3.10)  k(H)  - (||  H||  + ||  I - H||  ) sup  IIS  ||  H*"1  T)  ||. 

II  k 1.0 

From  (3.8),  (3.9)  and  (3.10)  it  easily  follows  that 

(3.11)  IS  ||  jc  - a||  * C k(H)  c ||  a||  + 0(C2). 

k K J 

We  want  to  determine  when  (3.11)  is  sharp.  In  order  to  do  this  we  must 

assume  something  more  about  g^.  Recall  that  the  vector  is  the  rounding 

error  vector  at  the  kth  iterative  step^  see  (3.6)  and  (3.9).  In  general,  g^ 

can  have  an  arbitrary  direction  and  ||  ^ Hk  * 5^11  can  be  of  order  £k(H)c3||  a||. 

i"0 

To  make  this  point  clear  we  shall  assume  throughout  this  paper  that  {g^}  can 

be  any  sequence  satisfying  (3.9).  Thus  to  prove  the  sharpness  of  (3.11)  it 

is  enough  to  define  {g^}  such  that  g^  ■ (||  h||  + ||  I - h||  ) Cj||  a||  where 

* 

the  supremum  in  (3.10)  is  attainable  for  'H^. 

Note  that 

00 

(3.12)  k(H)  * (||  H||  + ||  I - H||  ) ^ ||  H1!! 

i-0 

★ 

and  the  inequality  in  (3.12)  holds  for  a hermitian  H,  H ■ H . 

Comparing  (3.11)  with  the  definition  of  numerical  stability  (2.1)  we 
see  that  to  get  numerical  stability  of  the  successive  approximation  Iteration, 
k(H)  has  to  be  of  order  cond(A).  Thus  we  have  proven 
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Theorem  3.1 

If  (3.5)  holds  Chen  Che  successive  approximaCion  ICeraClon  given  by 

(3.12)  and  (3.3)  is  numerically  s Cable  iff 

(3.13)  k(H)  £ Cg  cond(A) 

where  c^  ■ c^(n)  and  k(H)  is  given  by  (3.10).  ■ 

In  Che  nexC  secCion  we  deCermine  for  which  Crans forma Cions  k(H)  is 
comparable  wiCh  cond(A).  We  wane  Co  end  Chis  secCion  by  showing  chac  for 
||  H ||  noC  Coo  close  Co  uniCy  we  geC  numerical  sCabillCy.  More  precisely  lee 
q € [0,1)  be  a number  noC  Coo  close  Co  uniCy  (q  £ .9,  say).  If  ||  H (|  £ q 
Chen  due  Co  (3.12)  k(H)  £ (2q+l)/(l-q)  and  (3.13)  holds  wiCh 
c^  ■ (2q4-l)/ t(l-q)  cond(A) ) £ (2q+l)/(l-q) . This  means  ChaC  Che  successive 
approximaCion  ICeraClon  is  always  numerically  sCable  for  a class  of  problems 
for  which  |J  H ||  £ q.  However,  usually  for  ill-condiCioned  problems  (for 
large  cond(A)),  some  eigenvalues  of  H have  moduli  close  Co  1 and  k(H)  is 
large.  FurChermore  we  shall  see  ChaC  even  for  well-condidoned  problems  1C 
can  happen  ChaC  k(H)  is  large  which  IndicaCes  an  unsCable  case  of  Che  succes- 
sive approximaCion  ICeraClon. 
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4.  EXAMPLES  OF  NUMERICAL  STABILITY 

In  this  section  we  consider  some  examples  of  transformations  from 
Ax  “b  to  x a Hx  + g and  we  find  conditions  assuring  numerical  stability. 

For  the  sake  of  simplicity  we  assume  throughout  this  section  that  A is 
a hermitian,  positive  definite  matrix  and  A has  a form 

(4.1)  A “ I - B 

where  B is  hermitian  and  has  zero  diagonal  elements.  Furthermore  we  assume 
that  ||  A ||  < 2.  Let  and  X be  the  smallest  and  the  largest  eigenvalue 

of  A.  Thus  0 < s 1 and  1 £ Xm^  s 2.  Note  that  cond(A)  ■ X . 

Example  4 . 1 Jacobi  Iteration 

In  this  case  H 3 B and  h = b.  Thus  assumption  (2.5)  holds  for  any 
reasonable  algorithm  for  computing  Hx^  + h.  Since  H ■ I - A is  hermitian 
then 

||H||  - <?(H)  - •<(!  - Xatn,  - 1)  < 1. 

Note  that  c(H)  is  close  to  1 if  X^  Is  close  to  zero  (which  means  that  the 

problem  is  ill-conditioned)  or  X is  close  to  two  which  can  happen  even 

max  r 

for  well-conditioned  problems. 

From  (3.12)  we  get 
(7(H)  + X 

(4‘2)  k(H)  - -I"-  o(hT  • 

In  general  k(H)  can  considerably  exceed  the  condition  number  cond(A)  even 
for  very  small  n.  For  Instance  let  n = 3 and 


I 


£-'i 

tv! 
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(4.3)  A ■ I a 1 a I 


, 0 < a < 1/2, 


whose  eigenvalues  are  1 - a,  1 - a and  1 + 2a,  see  Young  [71,  p.  111].  We 
have  a(H)  “ 2a  and 


k(H)  " T '-"If*  c°nd (A)  - * 


Thus 


lixn  k(H)  =*  +®  and  11m  cond(A)  “ 4 
a- 1/2“  a- 1/2 


which  means  that  (3.13)  does  not  hold  for  values  of  a close  to  l/2.  We  per- 
formed some  numerical  tests  on  the  PDP-10  computer  where 


£ - 3xl0”9  with  a = [1,1, 1]T  for  a - j - l(f  *,  1 - 2,3,4  and  5. 


The  best  computed  results  had  relative  error  of  order  10  which  confirms 
theoretical  considerations.  Thus  Jacobi  iteration  for  very  well-conditioned 
system  (4.3)  with  the  value  of  a close  to  l/2  is  numerically  unstable. 

To  assure  that  k(H)  is  of  order  cond(A)  we  have  to  assume  something 
more  concerning  the  eigenvalues  of  A. 


Theorem  4.1 


Jacobi  iteration  is  numerically  stable  for  A “ A >0  and  A is  of  the 
form  4.1  iff 


(4.4) 


max 


c6  " c6(n)* 


1 


i 
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Proof 

Assume  that  (A .4)  holds.  Consider  two  cases. 

C*S a_I.  Let  1 - Xmln  * X^  - 1.  Then  k(H)  - (1  - Xmin  + 2 cond(A) 

and  (3.13)  holds  with  c5  - 2. 

.U-  Let  1 - X^  < ^ - 1.  Then  k(H)  -(2  - l)/(2  - X^)  . But 

from  (4.4)  we  have  l/(2  - X ) s c6/Xmln  and  k(H)  52  cond(A)  and  once 
more  (3.13)  holds  with  c^  “ 2 c^. 

The  necessity  of  (4.4)  easily  follows  from  the  above  example  (4.3)  with 

a -♦  l/2-.  Since  X , » 1 - a A l/2  and  X - 1 + 2a  * 2,  the  lefthand  side 
' min  max 

of  (4.4)  tends  to  infinity  as  a tends  to  l/2’  which  causes  instability  of 
Jacobi  iteration.  ■ 

Note  that  if  A has  Property  A or  equivalently  B has  the  form 


* 

i? 

i 


where  0^  and  O2  are  square  mull  matrices  (see  Young  [71,  p.  42])  then 

X , - 2 - X and  (4.4)  holds  with  c,  ■ 1.  Thus  we  get 

min  max  6 


Corollary  4.1 

If  A - A*  > 0 and  A has  the  form  4.1  and  Property  A then  Jacobi  Iteration 
is  numerically  stable.  ® 


Example  4.2  Richardson  Iteration 
In  this  case 

(4.6)  H - I - c A and  h - -c  b 


11 


X - X . 

where  c - 2/(Xmin  + X^) . Then  ||h||  - a(H)  - >-atn  and 


3 X - X max  «in 

k(H)  ^IPX s 2 cond(A)  which  due  to  (3.13)  proves 


Theorem  4.2 

* 

If  A ■ A >0  then  Richardson  iteration  is  numerically  stable. 


Example  4.3  Gauss- Seidel  Iteration 


Assume  that  A ■ I - B has  Property  A.  Thus 


B - L + 


■•(? ') 


where  L and  U are  strictly  lower  and  strictly  upper  triangular  matrices.  Gauss- 


Seidel  Iteration  is  defined  by 


H = (I  - L"1) 


"C 


h - (I  - L"1)  b. 


It  is  easy  to  verify  that 


..  „k  /»!  F <F*  F>k'"\  ..  k,.  „,k-l 

•6>  H - I * 1 • II  H ||  - a(B) 

\0  (F  F)K  J 


From  (3.12)  we  get 


+ a (B)  , Vk  l 1 


k(H)  * (1  + 2 owJl  + c2(B))(1  + Jl  + ct2(B)  a(B)2k’1)  * 

Lu i 

k-1 

s (1  + 2 Ji  )(1  + / 2 * (1  - a(B))”1) . 


I 


Since  a(B)  ■ 1 - Xmin  we  have  (1  - a(B)) 


cond(A)  / X £ cond(A) . This 


-l 

proves  that 

k(H)  £ cond(A)  with  Cj  s (1  + iJi  ) (1  + Jl  / 2)  * 6.5. 

Hence  we  have  proven 
Theorem  4.3 

* 

If  A ■ A >0  and  A has  the  form  4.1  and  Property  A then  Gauss* Seidel 
iteration  is  numerically  stable.  ■ 

Example  4.4  Successive  Overrelaxation  Iteration  (SOR) 

Assume  that  A ■ I - B has  Property  A.  SOR  iteration  is  defined  by 

H - (I  - w L)-1(w  U + (l-w)I), 

(4.9)  , 

h ■ w(I  - w L)  b 

where  the  optimal  w is  given  by 


■ -yuc — 
1 + m ( 


a2  ( B) 


It  is  easy  to  verify  that 


Furthermore  from  Young  [71,  p.  248]  it  follows  that 


||  H||  - ok(H)  {k(aOI)1/2  + o(H)"1^2)  + ,^2(a( H)1^2  + c(H)*1/2)2  + l) 
s 2.3  k ok(H)(a(H)1//2  + a(H)"1//2) 


which  yields 


k(H)  s (1  + 2 ||  H||  )[1  + 2.3(<t(H)1/2  + a(H)“ 
10.2(1  + 4.6(1  - cr(H))”2)  • 


liHiii  i 


k <J(H)k]  s 
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Since 

(1  - C(H))‘2  - cond(A) (1  + cond(A)’1/2)4  / 16 

we  have  k(H)  £ cond(A)  with  £ 10.2  * 5.6  • 57.  However  if  cond(A)  is 
large  then  is  less  than  4.  Hence  we  have  proven 

Theorem  4.4 

* 

If  A ■ A >0  and  A has  the  form  4.1  and  Property  A then  SOR  iteration 
is  numerically  stable.  ■ 


■Hi 
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5.  GOOD- BEHAVIOR  OF  SUCCESSIVE  APPROXIMATION  ITERATIONS 

Recall  that  we  transform  the  linear  system  Ax  + g ■ 0 to  an  equivalent 
system  (I  - H)  x ■ h which  is  solved  by  constructing  (x^)  such  that 

(5.!>  xkfl-Hxk  + h. 

He  define  two  different  sequences  of  residuals  vectors,  A(x^  - nr)  for  the 
original  system  and  (I  - H)  (xk  - or)  for  the  transformed  one.  Let 

(5.2)  rfc  - M(xk  - or) 

where  M ■ A or  M ■ I - H.  We  want  to  verify  good-behavior  of  the  successive 
approximation  iteration  with  respect  to  A or  I - H.  Due  to  (2.2)  we  need  to 
prove  that 

(5.3)  TIS  (I  r ||  < C c II  Mll  II  “II  + OCC2) 

k K 

for  a constant  ■ C2O*).  From  (3.8)  we  get 

k 

(5.4)  rk+1  - M Hk+1  eQ  + ^ M Hk_i 

i-0 

where  ^ is  given  by  (3.6)  and  (3.10). 

Let  be  a sequence  such  that  ||  T]^||  £ 1.  Define 

k 

(5.5)  k(M.B)  - (II  h||  + II  I - h||  ) sup  !E||  * Bk"1  \||  . 

II  V51  k i-0 
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Since  (5.6)  is  sharp,  (5.3)  yields 


Theorem  5 . 1 


If  (3.5)  holds  then  the  successive  approximation  Iteration  is  well-behaved 
with  respect  to  M iff 


(5.7)  k(M,H)  s c6  ||  M 


where  c^  ■ c^(n) 


Remark  5.1 


Ue  showed  in  Section  3 that  ||  H ||  Sq  where  q is  not  too  close  to  unity 
implies  numerical  stability  of  the  successive  approximation  iteration.  It 
is  also  obvious  that  ||h  ||  £ q yields  good-behavior  since 


k(M,H)  INI 

and  (5.7)  holds  with  c^  ■ (2q  + 1)  / (1  - q). 

In  general,  it  is  rather  hard  to  evaluate  k(M,H).  However  for  many 
cases  it  is  enough  to  know  some  bounds  on  k(M,H). 


Lenina  5.1 

Let  X ^ 0 be  an  eigenvalue  of  H,  H?  “ X?  with  ||  ?||  “ 1*  Then 
(5.8)  k(M,H)  * x _^X|-  ||  M5 ))  . 


Proof 


Define  T)  - r ?.  Then 

1 W 


I 


-16- 


L M 1>k"1  \ 


Xk  1 


which  proves  (5.8). 


Lemma  5.2 


Let  M - I - H.  If  an  iteration  is  well-behaved  then 


(5.9)  max  I * c ||l-H| 

X€spect (H)  1 - 6 


Proof 


From  Lemma  5.1  and  5.7  we  get 


1 .Vl  II M 5 II"  ^H"[x)  s h(M,H)  Sc6  ||  I — H || 

for  any  eigenvalue  of  H which  proves  (5.9).  ■ 

Lemma  5.2  states  a necessary  condition  for  good-behavior  with  M * I - H 
which  means  that  |X|  - 1 implies  X - 1 for  any  eigenvalue  of  H. 


Lemma  5.3 

★ 

Let  M • I - H and  H ■ H . Then  an  iteration  is  well-behaved  iff 


(5.10)  >n  j-=4jr  ‘ c ||  I - H||  . 
X€spectfH)  1 " X 


X€spect(H) 


Proof 

* * 

Let  H - U D U where  U U - I and  D - diag(Xj Xr) . Let 

• U*  Then 

k k k 

, M Hk_i  \ - U(I  - D)  D11"1  U*  \ - U[ (1  - X.)  , X^1 

1-0  1 i'S'O  1 1 i-'O  1 1 


n - x ) xk"x 

...,(!  V^n  n J 


(5.11)  k(I  - H,H)  5 3 max 

j 


1 - \ 


Since  (5.11)  is  sharp,  (5.10)  is  proven.  • 

Note  that  (5.10)  means  that  H does  not  have  eigenvalues  close  to  -1. 

We  end  this  section  by  showing  that  for  H “ H it  is  often  possible  to 

redefine  the  transformed  system  such  that  (5.10)  holds  and  yields  good-behavior 

2 

Multiply  (1  - H)x  = h by  I + H.  Then  x « H x + (I  + H)  h and  we  can  iterate 

(5.12)  xk+1  =■  + (I  + H)  h. 

We  shall  call  the  iteration  (5.12)  as  the  modified  successive  approximation 

2 2 * 

iteration.  Note  that  H ■ [H  ] i 0 and  the  lefthand  side  of  (5-1 0)  is  equal 
to  unity.  Thus,  if  ||  I - H ||  is  not  too  small,  ||  I - H ||  i c^  for  c^  s .1, 
say,  then  we  get  good-behavior.  Hence  we  have  proven 


approximation 


Lemma  5.4 


If  H » H and  ||  I - h ||  S c^  > 0 then  the  modifed  successive 

2 2 

(5.12)  is  well-behaved  for  M ■ I - H and  c^  ■ l] <T  ' 


iteration 
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6.  EXAMPLES  OF  GOOD- BEHAVIOR 


As  in  Section  4 we  assume  that  A ■ A >0.  Except  Example  6.2  we  addi- 
tionally assume  that  A has  Property  A,  see  (4.1)  and  (4.5). 


Example  6.1  Jacobi  Iteration 


In  this  case  H ■ I - A is  hermit ian  and  \ - 2 - A . Apply  1 

min  max 

with  A ■ 1 - X for  M " A and  next  M ■ I - H.  In  both  cases  we  get 
max 


k(M,H)  2 


2 - X 


cond(A) 


which  shows  that  Jacobi  iteration  is  not  well-behaved. 

2 

For  the  modified  Jacobi  iteration  (5.12)  let  A «■  (1  - A ) . Then 


MA.H2)  * 952 . 1 a con^a 

* - <>  * 


which  contradicts  good- behavior.  Finally  notice  that 


Hi  - h2!!-  max  X(2  - X)  “ c?. 

X6spect(A) 


If  one  of  eigenvalues  of  A is  close  to  unity  then  c^  ■ 1 which  yields  good- 
behavior  of  the  modified  Jacobi  iteration  for  M “ I.  Thus  we  get 


Theorem  6.1 

Jacobi  iteration  is  not  well-behaved  for  M ■ A or  M ■ I ■ H.  The  modified 
Jacobi  Iteration  is  not  well-behaved  for  M ■ A and  it  is  well  behaved  for 


M ■ I - H whenever  A has  an  eigenvalue  close  to  unity. 


From  (4.6)  we  have 


(I  - H)H 


■c 


* * k- 1 ' 

F (I  - F F)  (F  F) 


(I  - F*  F) (F* 


r * * 

0.  F(I  - F F) (F 


F)k-1\ 

»k  /’ 


Vk  i 1. 


T T 

n\ 1 /o')  t 

We  estimate  k(M,H)  from  (5.5).  Let  ] . Then 

(I  - H)Hk-i  T)  - [w<1)T,  w^2)T]T 

i-0  IKK 

where 

V1'  - F(I  - F*  F)  ^ (F*  F)k-1'1  nj2)  + 1^l)  - F l£2)" 


v<2>  - (I  - F*  F)  ^ (F*  F)k-‘  f > . 


Since  F F is  nonnegative  definite  then  repeating  the  proof  of  Leona  5.3  it  is 


easy  to  verify  that 


lim  llw^Hs  (2  ||  F ||  + 1)  lim  ||  \||  * 3, 


lim  ||«k||<l 
k K 


which  yields 


k(I  - H,  H)  s (||  H||+  ||  I - H||)  J)+i  * * 6.3. 


Due  to  the  form  of  A H it  can  be  verified  that 


k(A,  H)  S 6. 


Since  ||  A ||  and  ||  I - h||  are  both  not  less  than  unity  we  finally  get 


t 


k(I  - H,  H)  5 2 TlO  ||  I - H||,  k(A,H)  S 6 ||  A || 


which  due  to  (5.7)  proves  good- behavior.  Hence  we  have  shown 


Theorem  6.3 

Gauss-Seidel  iteration  is  well-behaved  for  M = A and  M ■ I - H. 


Example  6. A SOR  Iteration 


In  this  case 


H - (I  - wL)  (wU  + (1  - w)  I) 


where  w=,2/(l+-/l-c  (B))  and  A a I - B. 

Let  be  an  eigenvalue  of  B.  Then  the  eigenvalues  of  H are  equal  to 


X - 7<w2  u2  - 2(w  - 1))  + 1 7(4 (w  - 1)  - w2  a2)  v2  u2 


where  i = J-l,  see  Young  [71,  p.  203].  From  this 


| X | * w - 1 * <r(H)  and  1 1 - \|  ■ w .j\  - u 


We  apply  Lenina  5.1  with  M * I - H and  next  M 3 A.  Then 


kd  - h,h)  - i-.-;-(Sr 2 \ ^ 


It  is  known  that  u "0  is  an  eigenvalue  of  B whenever  the  size  of  the  problem  n 


is  odd  which  yields 


k(I  - H,H)  * ~ . 'fcond(A) . 


Hence  SOR  is  not  well-behaved  for  M - I - H. 


Now  let  M ■ A,  u ■ - a(B)  and  let  5 be  an  eigenvector  associated  with 
A - w - 1,  ? - [5^,  §£]T,  ||  C||  - 1.  From  Young  [71,  p.  237]  It  follows 

.rfT  .-l/2  T /o\\r»T  *-l/2  rT,T 


k(A,H)  ■ (1  - cr(H))"1 1|  A[?J,  A"l/2  ?J]T  - 


- A[0T,  CA'1/2  - 1)  eJ]T||  * 


£ ,/cond(A)  [1  + ct(B)  - 2 (ct(H)”V*  - 1)  ] 


which  tends  to  Infinity  as  cond(A)  does.  Hence  SOR  Is  also  not  well-behaved 
for  M ■ A.  Hence  we  have 


Theorem  6.4 

SOR  iteration  Is  not  well-behaved  for  M ■ I • H or  M ■ A. 
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